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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment See 37 CFR 1 .704(b). 

Status 

1)^ Responsive to communication(s) filed on 16 September 2007 . 
2a)D This action is FINAL. 2b)(3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-76 is/are pending in the application. 

4a) Of the above claim(s) 1 5-62 and 64-76 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E] Claim(s) 1-2,5.8-10,13-14.63 is/are rejected. 

7) E3 Claim(s) 3.4,67.11 and 12 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)13 The drawing(s) filed on 28 July 2004 is/are: a)G3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)13 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)EI All b)D Some * c)D None of: 

1 Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 



Response to Restriction Requirement 

1 . Applicant's election without traverse of Species 1 (claims 1-14 and 63) in the reply filed 
on September 19, 2007 is acknowledged. Claims 15-62, 64-76 are withdrawn from further 
consideration by the Examiner, 37 CFR 1.142(b), as being drawn to a non-elected invention. 



Double Patenting 

The nonstatutory double patenting rejection is based on a judicially created doctrine 
grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or 
improper timewise extension of the "right to exclude" granted by a patent and to prevent possible 
harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection 
is appropriate where the conflicting claims are not identical, but at least one examined 
application claim is not patentably distinct from the reference claim(s) because the examined 
application claim is either anticipated by, or would have been obvious over, the reference 
claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re 
Goodman, 1 1 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re LongU 759 F.2d 887, 225 
USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re 
VogeU 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 
USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may 
be used to overcome an actual or provisional rejection based on a nonstatutory double patenting 
ground provided the conflicting application or patent either is shown to be commonly owned 
with this application, or claims an invention made as a result of activities undertaken within the 
scope of a joint research agreement. 

Effective January 1 , 1994, a registered attorney or agent of record may sign a terminal 
disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 
3.73(b). 



2. Claim 1 is rejected on the ground of nonstatutory obviousness-type double patenting as 
being unpatentable over claim 1 of U.S. Patent No. 7,321,690, in view of the article entitled 
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"Extraction of Unconstrained Caption Text from General-Purpose Video" by Crandall 
(hereinafter "Crandall"). 

Referring to claim 1 of the instant application, claim 1 of the '690 patent discloses all the 
claimed features except for the image binarization and character recognition part. 

Crandall discloses an image binarization part for comparing pixels in the character blocks 
with a pixel threshold, binarizing the pixels in the character blocks into a brightness value for a 
character pixel and a brightness value for a background pixel based on the comparison, and 
binarizing pixels in the background blocks into the brightness value for a background pixel; and 
a character recognition part for recognizing characters in the binarized image [pages 75-97. 
Crandall discloses several binarization techniques for binarizing pixels in an image into a 
brightness value for a character pixel and a brightness value for a background pixel by 
comparing pixel values with a pixel threshold. The result of one of these techniques is illustrated 
in figure 4.3.] 

Crandall also discloses a character recognition part for recognizing characters in a 
binarized image [pages 75-97]. 

Claim 1 of the '690 patent and Crandall are combinable because they are both concerned 
with character image processing systems. At the time of the invention, it would have been 
obvious'to a person of ordinary skill in the art to modify claim 1 in view of Crandall. The reason 
for doing so would have been to bridge the gap between character localization and recognition in 
order to meet the eventual goal of recognizing the text appearing in the image [Crandall, page 
75]. Therefore, it would have been obvious to combine claim 1 of the '690 patent with Crandall 
to obtain the invention as specified in claim 1 of the instant application. 
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Claim Rejections - 35 USC §103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3. Claims 1, 2, 63 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Crandall and Feng, U.S. Patent Application Publication No. 2004/0120598 
(hereinafter "Feng"). 

Referring to claim 1, Crandall discloses a device for recognizing characters in an image, 
comprising: 

an input part for receiving the image [pages 18-19]; 

an image binarization part for classifying the received image into character blocks and 
background blocks [pages 18-19], comparing pixels in the character blocks with a pixel 
threshold, binarizing the pixels in the character blocks into a brightness value for a character 
pixel and a brightness value for a background pixel based on the comparison, and binarizing 
pixels in the background blocks into the brightness value for a background pixel [pages 75-97. 
Crandall discloses several binarization techniques for binarizing pixels in an image into a 
brightness value for a character pixel and a brightness value for a background pixel by 
comparing pixel values with a pixel threshold. The result of one of these techniques is illustrated 
in figure 4.3.]; and 

a character recognition part for recognizing characters in a binarized image [pages 75- 

97]. 
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Crandall does not explicitly disclose a blurring decision part for calculating an average 
energy ratio of the character blocks and comparing the average energy ratio with a predetermined 
threshold to determine whether the received image is blurred. However, this feature was 
exceedingly well known in the art. For example, Feng discloses a blurring decision part for 
calculating an average energy ratio of the image blocks and comparing the average energy ratio 
with a predetermined threshold to determine whether the received image is blurred [pars. 25, 35- 
39. Note that a "blur indicator" is determined by calculating an average energy ratio of the 
image blocks (par. 35). The blur indicator is subsequently compared with a predetermined 
threshold to determine whether the image is blurred (pars. 36-39).]. 

Crandall and Feng are combinable because they are both concerned with processing 
digital images. At the time of the invention, it would have been obvious to a person of ordinary 
skill in the art to modify Crandall to include the blurring decision part of Feng. The reason for 
doing so would have been to detect and reduce any blur in the characters of the image. The 
detection and reduction of blur would have produced characters that are sharp and easier to 
recognize, thereby enhancing the character recognition process. Therefore, it would have been 
obvious to combine Crandall with Feng to obtain the invention as specified in claim 1 } 

Referring to claim 2, Crandall further discloses a block classification part for dividing the 
received image into blocks, and classifying the divided blocks into character blocks and 
background blocks [18-19]. 

1 The Examiner notes that in combining Crandall with Feng, it would have been logical to apply Feng's blur 
detection and reduction process prior to Crandall's binarization and character recognition process, in order to ensure 
that the characters are sharp and easily recognizable. Consequently, by applying Feng's blur detection and reduction 
technique, any blur in the image would have been reduced prior to binarization and therefore, the binarization 
process would have been applied to an image that is not blurred. 



Application/Control Number: Page 6 

10/798,860 

Art Unit: 2624 

Feng further discloses a character block energy calculation part for calculating an average 
energy ratio of the image blocks and a blurring detection part for comparing the average energy 
ratio with a predetermined threshold, and determining whether the image is blurred based on the 
comparison [pars. 25, 35-39]. 

Referring to claim 63, see the rejection of at least claim 1 above. The method of 
performing the steps recited in claim 63 is inherent in the system disclosed by Crandall and 
Feng, as recited in claim 1 . 

4. Claims 5, 8, 9, 13 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Crandall, Feng, and the article entitled "Adaptive, Quadratic, Preprocessing of 
Document Images for Binarization" by Mo et al. (hereinafter "Mo"). 

Referring to claim 5, Crandall further discloses a block classification part for dividing the 
image into blocks, and classifying the divided blocks into character blocks and background 
blocks [pages 18-19]; 

a binarization part for generating a pixel threshold for distinguishing between character 
pixels and background pixels in the character blocks; and for comparing pixels in the character 
blocks with the threshold, binarizing the pixels into a first brightness value for a character pixel 
and a second brightness value for a background pixel based on the comparison, and binarizing 
pixels in the background blocks output from the block classification part into the second 
brightness value [pars. 25, 35-39. Note that a "blur indicator" is determined by calculating an 
average energy ratio of the image blocks (par. 35). The blur indicator is subsequently compared 
with a predetermined threshold to determine whether the image is blurred (pars. 36-39).]. 
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Crandall does not explicitly disclose an edge enhancement part for enhancing edges of 
the character blocks using a relationship between neighboring pixels in the character blocks. 
However, this feature was exceedingly well known in the art. For example, Mo discloses an 
edge enhancement system for enhancing edges of character blocks using a relationship between 
neighboring pixels in the character blocks [pages 994-999, sections II and III]. 

Crandall, Feng, and Mo are combinable because they are all concerned with processing 
digital images. At the time of the invention, it would have been obvious to a person of ordinary 
skill in the art to modify Crandall and Feng to include the edge enhancement part of Mo. The 
reason for doing so would have been to enhance the character recognition process by providing 
characters having better defined edges and separation [Mo, page 993, left column]. Therefore, it 
would have been obvious to combine Crandall and Feng with Mo to obtain the invention as 
specified in claim 5. 

Referring to claim 8, Mo further discloses: 

a first threshold calculation part for calculating a first threshold for classifying pixels in 
into character pixels and background pixels [page 992, section L Note that the single threshold 
used in the global thresholding process is construed as the first threshold.]; 

a mean computation part for classifying pixels into character pixels and background 
pixels on the basis of the first threshold [page 992, section I. Note that a two output level image 
is produced based on the threshold, one level representing characters and the other level 
representing background.], and calculating mean brightness values (average gray-level) for 
character pixels and background pixels in the image [page 995, top right column]; 
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a normalization part for normalizing the pixels in the character block using a mean 
brightness value for a character pixel and a mean brightness value for the background pixel 
output from the mean computation part so that the character pixels have a value close to 1 T and 
the background pixels have a value close to '0' [page 995]; 

a quadratic operation part for performing a quadratic operation on the normalized 
character block to enhance edges of the character block and reduce noises of the character block 
[pages 994-995. Note that the normalized image is sent to the adaptive preprocessor, where a 
quadratic operation is performed to enhance edges and reduce noise.]; and 

a second threshold calculation part for normalizing the first threshold to calculate a 
second threshold for classifying the pixels into character pixels and background pixels, and 
outputting the second threshold as a threshold for the binarization part [page 996, top left 
column. Note that the global threshold value is normalized and output as a second threshold for 
the binarization process]. 

Referring to claim 9, Crandall further discloses: 

a block classification part for dividing the image into the blocks, and classifying the 
divided blocks into character blocks and background blocks [pages 18-19]; 

a block growing part for growing the character blocks to restore a block including 
character pixels, incorrectly classified as a background block, to a character block [pages 20-22]; 

a binarization part for comparing pixels in the character blocks with a threshold generated 
for distinguishing between character pixels and background pixels, binarizing the pixels into a 
first brightness value for a character pixel and a second brightness value for a background pixel 
according to the comparison result, and binarizing pixels in the background pixels into the 
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second brightness value [pages 75-97. Crandall discloses several binarization techniques for 
binarizing pixels in an image into a brightness value for a character pixel and a brightness value 
for a background pixel by comparing pixel values with a pixel threshold. The result of one of 
these techniques is illustrated in figure 4.3.]. 

Crandall does not explicitly disclose an edge enhancement part for enhancing edges of 
the character block output from the block growing part using a relationship between neighboring 
pixels in the character blocks. However, this feature was exceedingly well known in the art. For 
example, Mo discloses an edge enhancement system for enhancing edges of character blocks 
using a relationship between neighboring pixels in the character blocks [pages 994-999, sections 
II and III]. 

Crandall, Feng, and Mo are combinable because they are all concerned with processing 
digital images. At the time of the invention, it would have been obvious to a person of ordinary 
skill in the art to modify Crandall and Feng to include the edge enhancement part of Mo. The 
reason for doing so would have been to enhance the character recognition process by providing 
characters having better defined edges and separation [Mo, page 993, left column]. Therefore, it 
would have been obvious to combine Crandall and Feng with Mo to obtain the invention as 
specified in claim 9. 

Referring to claim 13, Mo discloses a noise reduction part for reducing noise of the 
received image and outputting the noise-reduced image to an image binarization part [pages 994- 
995. Note that the adaptive quadratic preprocessor performs the dual function of edge 
enhancement and noise reduction on the image before it is sent to the image binarization part.]. 
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5. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over the combination 
of Crandall, Feng, Mo, and Laumeyer et al., U.S. Patent No. 6,449,384 (hereinafter 
"Laumeyer"). 

Referring to claim 10, Crandall further discloses a block growing part for growing the 
character blocks to restore a block including character pixels, incorrectly classified as a 
background block, to a character block, as explained in claim 9 above. However, Crandall does 
not explicitly disclose that the block growing part includes a dilation and closing part for dilating 
and eroding respectively, the character block. 

Laumeyer discloses a block growing part that includes a dilation part and a closing part 
for dilating and eroding respectively image blocks [col. 6, 11. 14-60]. 

At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to modify Crandall's block growing part in view of Laumeyer' s teachings. The reason for 
doing so would have been to provide the capability of rapidly analyzing images to detect and 
classify objects of interest, thereby enhancing the character extraction process [Laumeyer, 
abstract]. Therefore, it would have been obvious to combine Crandall, Feng, and Mo with 
Laumeyer to obtain the invention as specified in claim 10. 

6. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over the combination 
of Crandall, Feng, Mo, and Chun, U.S. Patent No. 5,949,916 (hereinafter "Chun"). 

Referring to claim 14, Mo further discloses that the noise reduction part comprises a 
direction filter [page 995, left column], but does not explicitly disclose that the filter is a Lee 
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filter. However, Lee filters were exceedingly well known in the art. For example, Chun 
discloses a Lee filter for reducing noise in an image [col. 1, 11. 27-35]. 

It would have been obvious to include Chun's Lee filter in Crandall, Feng and Mo. The 
reason for doing so would have been to enhance the flexibility of the noise reduction process by 
providing a filter that has weights that vary depending on the variance of noise in the image. 

Allowable Subject Matter 

7. Claims 3-4, 6-7, 1 1-12 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Charles Kim whose telephone number is 571-272-7421. The 
examiner can normally be reached on Mon thru Thurs 8:30am to 6pm and alternating Fri 9:30am 
to 6pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Charles Kim 
Patent Examiner 
Art Unit 2624 
chongr.kim@uspto.gov 

January 9, 2008 




